Several leading technology organizations are investing in building technologies that sell “software-as-a-service”. Such services provide access to shared storage and/or computing resources to clients or subscribers. Within multi-tier e-commerce systems, combinations of different types of physical and logical resources may be allocated to subscribers and/or their applications, such as whole physical or virtual machines or storage devices, key-addressable object storage, CPUs, memory, network bandwidth, I/O capacity, or bundled resources such as database servers, scientific computation clusters, and the like.
For many storage services, the significant selling points of the service may include very high levels of data durability, suitable for enterprise-level mission-critical applications. Some network-accessible storage services may be designed to support data durability levels of 99.99999999% or higher, for example. Such high levels of durability may be difficult for businesses and individuals to support economically on their own premises, making the prospect of using the network-accessible storage service extremely attractive, at least for the more important or valuable portions of client data.
One straightforward approach to supporting high durability involves storing redundant copies of objects. For example, writes to storage objects may typically be replicated at several storage devices in such environments, often at different physical locations. In order to reduce storage costs while maintaining high durability levels, sophisticated algorithms may be used that divide a given storage object into smaller partitions or segments and then distribute encodings of the partitions among numerous storage devices. Such approaches, which can include various types of erasure coding techniques, may provide similar durability to simpler replication strategies, while requiring less storage than the simpler replication strategies. However, in high-durability storage systems that rely on encoding partitions of storage objects, the costs of decoding and re-assembling the encoded partitions may become significant, at least for some types of applications.
While embodiments are described herein by way of example for several embodiments and illustrative drawings, those skilled in the art will recognize that embodiments are not limited to the embodiments or drawings described. It should be understood, that the drawings and detailed description thereto are not intended to limit embodiments to the particular form disclosed, but on the contrary, the intention is to cover all modifications, equivalents and alternatives falling within the spirit and scope as defined by the appended claims. The headings used herein are for organizational purposes only and are not meant to be used to limit the scope of the description or the claims. As used throughout this application, the word “may” is used in a permissive sense (i.e., meaning having the potential to), rather than the mandatory sense (i.e., meaning must). Similarly, the words “include,” “including,” and “includes” mean including, but not limited to.